{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "本文的文字及图片来源于网络,仅供学习、交流使用,不具有任何商业用途,版权归原作者所有,如有问题请及时联系我们以作处理。\n",
    "\n",
    "\n",
    "身边总是不经意间能看到词云的效果图。本章介绍python实现词云的方法。效果如下：\n",
    "\n",
    "![](https://pic2.zhimg.com/v2-61ff642353b0c9d998c44d70e69f7eb1_b.jpg)\n",
    "\n",
    "(词云--出现频率越多的词，字体越大 )\n",
    "\n",
    "## 工具篇\n",
    "\n",
    "1、安装可视化库\n",
    "\n",
    "![](https://pic3.zhimg.com/v2-530fc6ba189849fe89d51c7f6e2dbe1e_b.jpg)\n",
    "\n",
    "( matplotlib 是专业的python可视化库，可用来生成各种各样的图表。也是二级考试里面推荐的可视化库。)\n",
    "\n",
    "2、安装词云库， 用来生成词云图片信息的\n",
    "\n",
    "![](https://pic3.zhimg.com/v2-3af9404152212e4f45defd050f4969aa_b.jpg)\n",
    "\n",
    "3、安装 “结巴” 库， 这个名字起的真接地气， 给开发者点个赞\n",
    "\n",
    "![](https://pic1.zhimg.com/v2-50a4904facbc8dd3c03cefc724f0f498_b.jpg)\n",
    "\n",
    "这个库用来解析中文，把一句话解析成一个个的词，\n",
    "\n",
    "我们中文不像英文每个词之间有空格。需要根据语义分析拆分成词组\n",
    "\n",
    "我们用《剑雨》的一段对话举例：\n",
    "\n",
    "```\n",
    "import   jieba #引入结巴库\n",
    "str='师傅，他为何说禅机已到,\\\n",
    "     佛祖点化世人讲究机缘,\\\n",
    "     禅机一过缘即灭矣,\\\n",
    "     而禅机未到虽点亦不中\n",
    "     我愿化身石桥又是何意'\n",
    "\n",
    "print(str)\n",
    "# 解析拆分词组  lcut的方法\n",
    "words = jieba.lcut(str)\n",
    "print(words)\n",
    "```\n",
    "\n",
    "效果如下：\n",
    "\n",
    "![](https://pic3.zhimg.com/v2-5e719a972476e71376f7e73c2d1533fe_b.jpg)\n",
    "\n",
    "（ 把完整的句子拆分成一个个的词，看着断断续续，可能这就是结巴的由来吧，【拆词】最难的这部分，结巴库都做好了，这也是python强大的原因吧，各式各样的库都有了 ）\n",
    "\n",
    "## 结巴库\n",
    "\n",
    "网上下载txt格式的《西游记》,\n",
    "\n",
    "下载完成后转换成utf-8格式再保存一下。\n",
    "\n",
    "转化方法：记事本另存为的时候，选择编码格式UTF-8\n",
    "\n",
    "![](https://pic4.zhimg.com/v2-e088e6ce7fa8ca0e2a37288a91e5059b_b.jpg)\n",
    "\n",
    "稍微、稍微、稍微列一下涉及到的知识点\n",
    "\n",
    "( 不要被吓到，不难，都是二级考试要求的内容 )\n",
    "\n",
    "1.  print的format的格式化输出\n",
    "2.  for 循环用法\n",
    "3.  文件的开打，读取，关闭，文件路径\n",
    "4.  数组，python不叫数组，叫list列表\n",
    "5.  python的字典类型 key:value,类似java的map类型或者json对象\n",
    "\n",
    "准备工作：\n",
    "\n",
    "1.  西游记和python文件可放到一个目录，引用起来方便，直接文件名就 ok，不用写路径了，\n",
    "2.  拷贝一个中文字体库，放到同目录下，也是为了引入方便，在Windows的C:\\\\Windows\\\\Fonts里面，随便找一个中文字体，就行，否则生成的图片都是乱码， （也可以用绝对路径直接指定）\n",
    "3.  准备几张图片，词云支持按指定的图片形状显示，比如心形，五角星，汽包，或者人像等等，但要求图片是白色背景\n",
    "\n",
    "![](https://pic4.zhimg.com/v2-c0757a3a128ca9147a832bb04ee753db_b.jpg)\n",
    "\n",
    "format 格式化输出：\n",
    "\n",
    "```\n",
    "#列标题 format\n",
    "print(\"{0:<5}{1:<8}{2:<5}\".format('序号','词语', '频率'))\n",
    "```\n",
    "\n",
    "{0:<5} ：\n",
    "\n",
    "0 表示序号：第一个参数，第一列，\n",
    "\n",
    "< 左对齐， > 右对齐\n",
    "\n",
    "5 代表宽度\n",
    "\n",
    "任务一：统计西游记里面出现频率最高的10个词：\n",
    "\n",
    "后面为了方便，我们把这个叫主代码。\n",
    "\n",
    "```\n",
    "#引入结巴库\n",
    "import  jieba\n",
    "#open 内置函数 不需要引用 直接使用\n",
    "#开打文件西游记和python 文件放到同一个目录，可直接引用不需要路径\n",
    "f = open('西游记.txt', 'r' ,encoding='utf-8'  )\n",
    "# 查看文件的编码格式\n",
    "print('文件的编码格式:'+f.encoding)\n",
    "#读取文件\n",
    "txt = f.read()\n",
    "#关闭文件，良好的习惯\n",
    "f.close()\n",
    "\n",
    "# 使用精确模式对文本进行分词\n",
    "#  使用结巴库把西游拆分成一个个的词组\n",
    "words = jieba.lcut(txt)\n",
    "\n",
    "# 通过键值对的形式存储词语及其出现的次数\n",
    "# 大括号表示 python的字典类型对应，\n",
    "# 键值对 key:value1 ,类似java的map对象和list\n",
    "counts = {}\n",
    "\n",
    "chiyun = []\n",
    "for word in words:\n",
    "     # == 1 单个词语不计算在内\n",
    "    if  len(word) < 2 :\n",
    "        continue\n",
    "    else:\n",
    "         # 遍历所有词语，每出现一次其对应的值加 1\n",
    "        counts[word] = counts.get(word, 0) + 1\n",
    "\n",
    "#将键值对转换成列表\n",
    "items = list(counts.items())\n",
    "\n",
    "# 根据词语出现的次数进行从大到小排序\n",
    "items.sort(key=lambda x: x[1], reverse=True)\n",
    "#列标题 format\n",
    "print(\"{0:<5}{1:<8}{2:<5}\".format('序号','词语', '频率'))\n",
    "#需要显示的范围  10即显示前10个，0到9\n",
    "for i in range(10):\n",
    "    word, count = items[i]\n",
    "    print(\"{0:<5}{1:<8}{2:>5}\".format(i+1,word, count))\n",
    "```\n",
    "\n",
    "效果如下：\n",
    "\n",
    "![](https://pic2.zhimg.com/v2-16126ae033af113fdd5b38eda7e38941_b.jpeg)\n",
    "\n",
    "行者 妥妥的第一主角。\n",
    "\n",
    "我预期的师徒四人后，就应该有观音\n",
    "\n",
    "我们看前20是否有观音，在主代码里的第39行代码for循环的范围改成20\n",
    "\n",
    "![](https://pic4.zhimg.com/v2-a38623677da943fb967115c785d85ce3_b.jpeg)\n",
    "\n",
    "还是没有观音， 放出前100,200,300 到500在看：\n",
    "\n",
    "终于出现了，在349位，重复61次，\n",
    "\n",
    "应该是有哪里问题，直觉判断不止61次重复\n",
    "\n",
    "![](https://pic4.zhimg.com/v2-923cb0a26cba4e007c57e9347aaad7f3_b.jpeg)\n",
    "\n",
    "任务二：统计四个字重新次数\n",
    "\n",
    "把上面主代码25行，改成小于4，即长度小于4个的词不统计\n",
    "\n",
    "效果如下：\n",
    "\n",
    "![](https://pic3.zhimg.com/v2-0c19200d270f8b4b73fd90801c1e0fee_b.jpg)\n",
    "\n",
    "预知后事如何，请听下回分解，‘下回分解’排第二合理。\n",
    "\n",
    "悟空每次打妖怪都要吹下牛，在洞口报个名：我是五百年前大闹天宫的齐天大圣，这两个词出现在前10 合理。\n",
    "\n",
    "## 词云库\n",
    "\n",
    "我们在梳理下流程：\n",
    "\n",
    "![](https://pic1.zhimg.com/v2-749e185e4bfd76dd580727399692ba1c_b.jpg)\n",
    "\n",
    "注意下：结巴库不是必须，主要是用来把句子拆分成词，如果你已经有统计好的词组，可直接用词云显示。\n",
    "\n",
    "试一个简单的：我们手工创建个词组，直接用词云显示出来。即绕开结巴库了\n",
    "\n",
    "```\n",
    "from wordcloud import WordCloud\n",
    "# python的可视化库，也是二级考试推荐的可视化库\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "str=['齐天大圣','大圣','大圣','八戒','嫦娥']\n",
    "#数组里面添加内容\n",
    "str.append('玉兔')\n",
    "str.append('女儿国')\n",
    "str.append('牛魔王')\n",
    "str.append('大圣')\n",
    "str.append('土地公公')\n",
    "str.append('小神仙')\n",
    "str.append('八戒')\n",
    "\n",
    "print(str)\n",
    "#join 函数 用斜杆拼接词组mask =maskph,\n",
    "#这里一定要join拼接一下 转成字符串\n",
    "text_cut  =  '/'.join(str)\n",
    "\n",
    "#看一下连接后的样子\n",
    "#关键点 text_cut 是词云要处理的内容\n",
    "print(text_cut)\n",
    "wordcloud = WordCloud(  background_color='white',font_path = 'msyh.ttc', width=1000, height=860, margin=2).generate(text_cut)\n",
    "# 显示图片\n",
    "plt.imshow(wordcloud)\n",
    "plt.axis('off')\n",
    "plt.show()\n",
    "```\n",
    "\n",
    "可观察下效果图：主要join后的输出，用/拼接成了一个字符串：\n",
    "\n",
    "![](https://pic2.zhimg.com/v2-2bd4ccd2c1e72fdeed2d45eb93230991_b.jpg)\n",
    "\n",
    "效果图：\n",
    "\n",
    "![](https://pic3.zhimg.com/v2-a5b0f87f6b8d8f3c933538c3b5006c0a_b.jpg)\n",
    "\n",
    "(大圣和八戒出现次数多，字体最大)\n",
    "\n",
    "## 词云图\n",
    "\n",
    "先把全代码放上，后面实例在解析：\n",
    "\n",
    "这段可先略过，下面直接看效果图：\n",
    "\n",
    "```\n",
    "#引入结巴库\n",
    "import  jieba\n",
    "#词云库\n",
    "from wordcloud import WordCloud\n",
    "# python的可视化库，也是二级考试推荐的可视化库\n",
    "import matplotlib.pyplot as plt\n",
    "from PIL import Image  #处理图片的\n",
    " #矩阵 好像也是协助处理图片的\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "\n",
    "#open 内置函数 不需要引用 直接使用\n",
    "#开打文件西游记和python 文件放到同一个目录，可直接引用不需要路径\n",
    "f = open('西游记.txt', 'r' ,encoding='utf-8'  )\n",
    "\n",
    "# 查看文件的编码格式\n",
    "print('文件的编码格式:'+f.encoding)\n",
    "\n",
    "#读取文件\n",
    "txt = f.read()\n",
    "\n",
    "#关闭文件，良好的习惯\n",
    "f.close()\n",
    "\n",
    "\n",
    "# 使用精确模式对文本进行分词\n",
    "#  使用结巴库把西游拆分成一个个的词组\n",
    "words = jieba.lcut(txt)\n",
    "\n",
    "\n",
    "# 通过键值对的形式存储词语及其出现的次数\n",
    "# 大括号表示 python的字典类型对应，\n",
    "# 键值对 key:value1 ,类似java的map对象和list\n",
    "counts = {}\n",
    "#数组对象  用来接收需要传递给词云的内容\n",
    "chiyun = []\n",
    "for word in words:\n",
    "     # == 1 单个词语不计算在内\n",
    "    if  len(word) < 2 :\n",
    "        continue\n",
    "    else:\n",
    "         # 遍历所有词语，每出现一次其对应的值加 1\n",
    "        counts[word] = counts.get(word, 0) + 1\n",
    "\n",
    "#将键值对转换成列表\n",
    "items = list(counts.items())\n",
    "\n",
    "# 根据词语出现的次数进行从大到小排序\n",
    "items.sort(key=lambda x: x[1], reverse=True)\n",
    "\n",
    "\n",
    "#列标题 format\n",
    "print(\"{0:<5}{1:<8}{2:<5}\".format('序号','词语', '频率'))\n",
    "\n",
    "\n",
    "#需要显示的范围  10即显示前10个，0到9\n",
    "for i in range(80):\n",
    "    word, count = items[i]\n",
    "    print(\"{0:<5}{1:<8}{2:>5}\".format(i+1,word, count))\n",
    "    chiyun.append(word)\n",
    "\n",
    "\n",
    " #print(chiyun)\n",
    "\n",
    "\n",
    "#加载图片信息\n",
    "maskph = np.array(Image.open('山东舰航母.png'))\n",
    "\n",
    "\n",
    "#join 函数 用斜杆拼接词组\n",
    "text_cut  =  '/'.join(chiyun)\n",
    "wordcloud = WordCloud(mask =maskph,  background_color='white',font_path = 'msyh.ttc', width=1000, height=860, margin=2).generate(text_cut)\n",
    "# 显示图片\n",
    "plt.imshow(wordcloud)\n",
    "plt.axis('off')\n",
    "plt.show()\n",
    "```\n",
    "\n",
    "效果1：全部显示\n",
    "\n",
    "即：结巴库处理好的词组，不做限制，全部送给词云显示：\n",
    "\n",
    "![](https://pic1.zhimg.com/v2-bd013c87ee0e7392a6c5069e3c38c7cc_b.jpg)\n",
    "\n",
    "显示二：限定内容显示\n",
    "\n",
    "比如改成 只输出前20个词：（显示密度会稀好多）\n",
    "\n",
    "即：结巴库处理好后，取前20个高频词传给词云显示：\n",
    "\n",
    "（ 这个二级教材里面没有哈 ）\n",
    "\n",
    "![](https://pic1.zhimg.com/v2-5178d36b1509bb4ff59e0a360769fca0_b.jpg)\n",
    "\n",
    "再试一下前20的 四个字的词：\n",
    "\n",
    "![](https://pic3.zhimg.com/v2-08142f909dfe97686956a7bfa0559bfa_b.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}